{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPU-Jupyter\n",
    "\n",
    "This Jupyterlab Instance is connected to the GPU via CUDA drivers. In this notebook, we test the installation and perform some basic operations on the GPU."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test GPU connection\n",
    "\n",
    "#### Using the following command, your GPU type and its NVIDIA-SMI driver version should be listed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mon Apr 26 13:59:53 2021       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                               |                      |               MIG M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0  On |                  N/A |\n",
      "|  0%   48C    P8     8W / 215W |    283MiB /  7974MiB |     11%      Default |\n",
      "|                               |                      |                  N/A |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                                  |\n",
      "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
      "|        ID   ID                                                   Usage      |\n",
      "|=============================================================================|\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now, test if PyTorch can access the GPU via CUDA:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "torch.cuda.is_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[name: \"/device:CPU:0\"\n",
       " device_type: \"CPU\"\n",
       " memory_limit: 268435456\n",
       " locality {\n",
       " }\n",
       " incarnation: 6507124110760788315,\n",
       " name: \"/device:XLA_CPU:0\"\n",
       " device_type: \"XLA_CPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 4922654194336399393\n",
       " physical_device_desc: \"device: XLA_CPU device\",\n",
       " name: \"/device:XLA_GPU:0\"\n",
       " device_type: \"XLA_GPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 1179884248341804191\n",
       " physical_device_desc: \"device: XLA_GPU device\"]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.python.client import device_lib\n",
    "print(tf.config.list_physical_devices('XLA_GPU'))\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.6378, 0.9107, 0.5509],\n",
       "        [0.4454, 0.1930, 0.4130],\n",
       "        [0.0074, 0.9115, 0.1397],\n",
       "        [0.4758, 0.0569, 0.8469],\n",
       "        [0.1666, 0.7414, 0.5519]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "import torch\n",
    "a = torch.rand(5, 3)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance test\n",
    "\n",
    "#### Now we want to know how much faster a typical operation is using GPU. Therefore we do the same operation in numpy, PyTorch and PyTorch with CUDA. The test operation is the calculation of the prediction matrix that is done in a linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "358 ms ± 72.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "H = x.dot(np.linalg.inv(x.transpose().dot(x))).dot(x.transpose())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "187 ms ± 40.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "# Calculate the projection matrix of x on the CPU\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) PyTorch on GPU via CUDA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.5021, 0.1432, 0.7876, 0.4729, 0.8067],\n",
      "        [0.7827, 0.3770, 0.8910, 0.3543, 0.3826],\n",
      "        [0.1806, 0.2917, 0.3224, 0.2717, 0.3795],\n",
      "        [0.5002, 0.2753, 0.5238, 0.0830, 0.9391],\n",
      "        [0.0774, 0.3479, 0.8384, 0.6825, 0.4502]], device='cuda:0')\n",
      "tensor([[0.5021, 0.1432, 0.7876, 0.4729, 0.8067],\n",
      "        [0.7827, 0.3770, 0.8910, 0.3543, 0.3826],\n",
      "        [0.1806, 0.2917, 0.3224, 0.2717, 0.3795],\n",
      "        [0.5002, 0.2753, 0.5238, 0.0830, 0.9391],\n",
      "        [0.0774, 0.3479, 0.8384, 0.6825, 0.4502]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 256, device=device) # directly create a tensor on GPU\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])\n",
    "    print(y.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.42 ms ± 223 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "# Calculate the projection matrix of x on the GPU\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exhaustive Testing on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "import torch\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 10, device=device) # directly create a tensor on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.6581, 0.1584, 0.1951, 0.7333, 0.2423],\n",
      "        [0.7057, 0.4649, 0.6851, 0.4686, 0.4990],\n",
      "        [0.0904, 0.3937, 0.1658, 0.3733, 0.8904],\n",
      "        [0.2138, 0.0609, 0.5688, 0.1917, 0.8857],\n",
      "        [0.2110, 0.6726, 0.2961, 0.3625, 0.0745]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # Here is the memory of the GPU a border. \n",
    "    # A matrix with 100000 lines requires 37 GB, but only 8 GB are available.\n",
    "    H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 7.5589e-04,  9.0142e-05,  1.7263e-04,  7.5191e-05,  1.2231e-04],\n",
      "        [ 9.0142e-05,  5.8283e-04, -2.2237e-04,  3.5647e-04, -2.3438e-05],\n",
      "        [ 1.7263e-04, -2.2237e-04,  8.8304e-04,  3.3944e-04, -4.9345e-05],\n",
      "        [ 7.5191e-05,  3.5647e-04,  3.3944e-04,  9.6286e-04, -1.4842e-05],\n",
      "        [ 1.2231e-04, -2.3438e-05, -4.9345e-05, -1.4842e-05,  1.0476e-03]],\n",
      "       device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    print(H[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 7.5589e-04,  9.0142e-05,  1.7263e-04,  7.5191e-05,  1.2231e-04],\n",
      "        [ 9.0142e-05,  5.8283e-04, -2.2237e-04,  3.5647e-04, -2.3438e-05],\n",
      "        [ 1.7263e-04, -2.2237e-04,  8.8304e-04,  3.3944e-04, -4.9345e-05],\n",
      "        [ 7.5191e-05,  3.5647e-04,  3.3944e-04,  9.6286e-04, -1.4842e-05],\n",
      "        [ 1.2231e-04, -2.3438e-05, -4.9345e-05, -1.4842e-05,  1.0476e-03]],\n",
      "       dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # This operation is difficult, as an symmetric matrix is transferred \n",
    "    # back to the CPU. Is possible up to 30000 rows.\n",
    "    print(H.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
